Specular objects that exhibit mirror-like reflectance have no appearance of their own, but rather distort a surrounding environment. Traditional surface recovery methods designed for Lambertian surfaces, such as structure from motion (SfM), stereo or multi-view stereo, can not be used directly for specular objects.
Shape from specular flow (SFSF) explores surface estimation by measuring dense optical flow of environment features as observed on a mirror under a known motion of environment, mirror, or camera. Much of the prior art work in SFSF assumes infinitesimal rotation of the environment, where in the forward flow equations linking the motion field, surface parameters and specular flow (SF) can be expressed as a partial differential equation (PDE).
However, conventional approaches do not extend to finite or large motion because large displacements cannot be incorporated into the PDE framework. Further, SFSF requires a dense optical flow, which is difficult to obtain for specular objects because SF exhibits certain undesirable properties, such as undefined or infinite flow, and one-to-many mappings.
Specular Flow verses Reflection Correspondences
FIG. 1 shows point P1 on surface S1, and point P2 on surface S2. The normals n, and curvatures of the surfaces at the two points are identical. Because the specular flow forward equations depend only on local normal and curvature, for the same infinitesimal environment motion, the flow will be identical for these two points.
However, in case of a finite or large environment rotation θ as shown, the observed flow depends not only on the local normal and curvature, but also on properties of the neighborhood. Notice that the actual flow observed at these two surface points is radically different because of the difference in their neighborhoods.
This dependence is not easily incorporated into the specular flow framework.
Qualitative Properties:
It has been shown show that local surface properties, such as concave or convexity, can be determined under motion of the observer without knowledge of the lighting. Stereoscopic images of specular highlights can be analyzed to shows that the disparity of highlights is related to its convexity or concavity.
Active Illumination
The structure of specular objects can be estimated using photometric stereo images. Caustics can be bused to determine if an image feature is real or a reflection. Then, surface recovery can be performed by tracking an unknown scene point. This property can then be used to estimate surface mesostructures at high resolution. Probes of a known surface and reflectance can determine the properties of an unknown surface.
Calibrated Environment
Surface reconstruction under known environment is well known. Local properties of a smooth surface can be determined up to third order by the observed projection of two calibrated lines. Multiple images of a calibrated pattern can be used to reconstruct the shape of the surface mirror. Structured highlighting can also be used to obtain the shape of specular objects.
Uncalibrated Environments
Specular flow can be significantly different from the motion field, especially near points of parabolic curvature. Specular path perturbation can be used for fast rendering of multiple inter-reflections.
Specular flow can also be determined for two known or three unknown rotations about arbitrary axes. Both of these approaches require a significant number of initial conditions (in terms of normals) to solve for the shape of the surface. Under a different parameterization of the surface, the estimation problem becomes linear. This is used to estimate the shape of the mirror from multiple specular flows without any additional information.
However, the SFSF formulation is inherently differential and does not extend to large motion.